This application relates generally to audible feedback from 3-dimensional (3-D) imagery, and more specifically to embodiments in which a surgeon receives audible feedback based on the location of a surgical instrument as mapped on reconstructed surgical views.
Stereotactic surgery is known in the art as a technique for localizing a target in surgical space. The use of stereotactic instrumentation based on tomographic imaging is conventional in surgery. When used for brain or other neurosurgery, such methods may involve attaching a headring apparatus to the patient""s skull, and then using conventional techniques to acquire imaging data where the data is space-related to the headring. A surgeon may use an arc system to relate the position of the head to the radiographic image. An indexing device, localizer structure or other fiducial apparatus is generally used to specify quantitative coordinates of targets (such as tumors) within the patient""s head relative to the fiducial apparatus. An exoscope may be used to aid in visualization during the procedure.
Current technology also allows use of a frameless system, to provide a visual reference in the operating room. Again with reference to brain surgery, fiducial markers are placed around the patient""s head so as to be apparent on the pre-operative magnetic resonance imaging (MRI) or computerized tomography (CT) scan. Techniques known in the art are used pre-operatively to localize the fiducial markers located on the patient, and a computer compares this information to that from the previous imaging. The actual location of the patient""s head may thus be registered to, and correlated with, the computerized three-dimensional reconstruction of the patient""s head.
As the surgery proceeds the surgeon can use the image guidance system to locate the surgical target and track a resection instrument""s position in space, relative to the target, based on the live-time recognition of fiducial markers located on the instrument itself. Such image guidance systems using visual feedback to the image are disclosed and discussed in more detail in U.S. Pat. No. 5,961,456, incorporated herein by reference. Embodiments disclosed in U.S. Pat. No. 5,961,456 allow the surgeon to observe a video monitor that projects an actual, real-time image of the surgical field and the instrument moving in space. Superimposed on that image is an augmented-reality image, derived from the pre-operative scan, disclosing the position of the target.
As the surgery proceeds, the surgeon can use the image guidance system to locate the surgical target. The same guidance system can localize in space the relation of the resection instrument to the target. While serviceable and useful for improved guidance for the surgeon, such prior art visual feedback systems require the surgeon periodically to re-orient his/her field of view from the surgical instrument and the patient to the monitor in order to track the instrument. Further advantages would be available if the surgeon were able to localize the target, and define its volume, while continuously viewing the surgical field.
A further variation on the above conventional technology is for the surgeon to perform frameless stereotactic surgery with the assistance of a microscope. The microscope assists enlarged viewing of the surgical field. In this application, the surgeon views a two-dimensional image from the pre-operative scan superimposed on a corresponding three dimensional volume within the surgical field seen directly through the microscope. Although helpful for fine and delicate surgical procedures on microscopic tumors, this technique has limited benefit since the field of view of the microscope is small and microscope programs may not be available at a particular institution. A system using pre-operative scans to guide the surgeon in both enlarged and unenlarged environments would be highly advantageous.
These and other needs in the art are addressed by a computer-based system generating audible feedback to assist with guidance of a trackable point in space. Surgical embodiments include generating audible feedback (to supplement visual and tactile feedback) to a surgeon moving the tip of a probe, for example, in a surgical field with respect to a volume of interest such as a tumor. Other surgical embodiments include generating audible feedback to assist with the precise insertion of a pedicle screw, inserting a biopsy cannula or electrode into the brain, and the like.
Exemplary surgical embodiments are based on correlating a three-dimensional (3-D) reconstruction of an anatomical object to real-time stereotactic surgery via a lexicon of audible signals. More specifically, these audible signals may be generated according to the position of the point of a probe or other instrument in surgical space, as seen by two or more cameras in relationship to a plurality of fiducial or fiduciary markers. The views of the cameras have previously been matched with a 3-D reconstruction of the surgical field or surgical target via the fiducial markers. Features of interest visible on the 3-D reconstruction, such as points, boundaries, planes and/or volumes, have previously been identified and correlated with predetermined audible feedback. As the surgeon moves a probe or instrument within the surgical field, a computer analyzes the current position of the probe or instrument in space with respect to the fiducial markers, and provides audible feedback as to its position relative to the predetermined features of interest identified on the 3-D reconstruction. The audible feedback may vary in numerous ways, including, but not limited to, changes in tone, volume, pattern and/or style, as the probe and/or instrument moves relative to the features of interest identified on the 3-D reconstruction.
Various illustrative embodiments of present invention provide a rich and comparatively unburdened sensory modality in the operating room and may be practiced independent of, or along with, conventional visually-oriented systems and techniques.
According to one aspect, therefore, a method for generating audible feedback is provided, the method comprising creating a computer-generated reconstruction of an overall volume, the reconstruction identifying one or more features of interest within the overall volume and creating a computer-generated real-time image of the overall volume, the computer-generated real-time image further including at least one trackable point, the trackable point(s) movable in real-time with respect to the overall volume. The method also comprises causing a computer to overlay the computer-generated real-time image and the computer-generated reconstruction with substantial spatial identity and substantial spatial fidelity, causing the computer to track the trackable point(s) with substantial positional fidelity to the real-time image, and creating computer-generated audible feedback, the feedback describing the position of the trackable point(s) with respect to the feature(s) of interest.
According to a second aspect, a method for generating audible feedback is provided, the method comprising creating a computer-generated 3-D reconstruction of an overall volume from a series of layered images, the reconstruction identifying one or more features of interest within the overall volume and creating a computer-generated real-time 3-D image of the overall volume by resolving digital output from at least two video cameras, the computer-generated real-time image further including at least one trackable point, the trackable point(s) movable in real-time with respect to the overall volume. The method also comprises causing a computer to overlay the computer-generated real-time image and the computer-generated reconstruction with substantial spatial identity and substantial spatial fidelity via reference to a system of fiducial markers, causing the computer to track the trackable point(s) with substantial positional fidelity to the real-time image, creating computer-generated audible feedback, the feedback describing the position of the trackable point(s) with respect to the feature(s) of interest, and selectably causing the audible feedback to vary in a predetermined fashion as a selected trackable point or points approach and/or withdraw from a predefined feature or features of interest.
According to a third aspect, a computer program product having computer-readable logic recorded thereon for generating audible feedback is provided, the computer program operable on a general purpose computer, the computer including a processor, a memory and a sound generator, the computer-readable logic comprising instructions for causing the computer to refer to a computer-generated reconstruction of an overall volume, the computer-generated reconstruction identifying one or more features of interest within the overall volume, and instructions for causing the computer to refer to a computer-generated real-time image of the overall volume, the computer-generated real-time image further including at least one trackable point, the trackable point(s) movable in real-time with respect to the overall volume. The computer-readable logic also comprises instructions for causing the computer, via reference to a system of fiducial markers, to overlay the computer-generated real-time image and the computer-generated reconstruction with substantial spatial identity and substantial spatial fidelity, instructions for causing the computer to track the trackable point(s) with substantial positional fidelity to the computer-generated real-time image, and instructions for causing the computer to generate audible feedback, the feedback describing the position of the trackable point(s) with respect to the feature(s) of interest.
It is, therefore, a technical advantage of various illustrative embodiments of the present invention that in surgical embodiments, audible sound may be used to provide positional feedback to a surgeon during a procedure. This allows the surgeon to receive positional feedback without the need for frequent reference to a monitor and/or a screen during surgery. Requiring the surgeon to look back and forth between a monitor and a surgical field will usually lengthen operating time. An increased risk of infection is often associated with increased operating time. In addition, the cost of a surgery diminishes with decreased operating time. Furthermore, using a monitor for orientation during the stereotactic procedure requires that a two-dimensional (2-D) monitor be related to a 3-D anatomical object. By using audio for guidance rather than solely video, the entire operation can be performed in a three-dimensional (3-D) environment.
A further technical advantage of various illustrative embodiments of the present invention is that, in surgical embodiments, increased precision during the operation is likely to result. This, in turn, is likely to result in a decreased possibility of neurological damage, scarring and/or adhesion formation.
A further technical advantage of various illustrative embodiments of the present invention is that the boundaries of an anatomical object such as a tumor may be discovered. Boundaries of anatomical objects such as tumors are often difficult to determine with a camera, magnification, and/or the naked eye. However, boundaries may be more accurately determined on a radiographic image. In surgical embodiments, various illustrative embodiments of the present invention enable audible feedback to be used to determine actual tissue boundaries with the help of a 3-D reconstructed image. As a result, the boundaries of an anatomical object may be more accurately determined. Other surgical embodiments provide audible feedback further correlated to the distances and orientation of the tip of the probe from the surgical feature(s) of interest.
A further technical advantage of various illustrative embodiments of the present invention is that the location of an anatomical object may be more easily and precisely determined. For example, various illustrative embodiments of the present invention may be useful for the insertion of pedicle screws, biopsy cannulas, electrodes and/or the like.
Although not discussed in detail, a similar coded audible signal can be used without the video display, for instance, on insertion of a biopsy needle or electrode with either a stereotactic frame or frameless guidance system. In addition, the audible signal can be used for other purposes besides surgery which may include engraving, carving, mounting, and/or inserting objects.
The foregoing has outlined rather broadly many of the features and technical advantages of the present invention in order that the detailed description of the present invention that follows may be better understood. Additional features and advantages of the present invention will be described hereinafter which form the subject of the claims of the present invention. It should be appreciated by those skilled in the art that the conception and the specific embodiments disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the present invention as set forth in the appended claims.